The present invention relates generally to digital film developing/processing and more particularly, to apparatus and methods for timed-scan image stitching and noise reduction.
Digital film processing (xe2x80x9cDFPxe2x80x9d) promises several advantages over traditional xe2x80x9cdarkroomxe2x80x9d film development techniques, including more accurate image reproduction, defect correction, and image processing, among others
A DFP system for capturing an image recorded on photographic film is illustrated in FIG. 1, which figure is taken from FIG. 9 of U.S. Pat. No. 5,519,510. As shown, conventional color photographic film 101 typically includes three film layers, each of which is constructed to record different primary color information (i.e. red, green and blue color information) of a source image during picture taking.
In one aspect of the ""510 patented invention, the DFP system captures image information from film 101 by projecting infrared light 11 and 12 at and correspondingly scanning reflected and/or transmitted light from successive portions while the film is being developed with a monochromatic developer. A complete image dataset is formed by capturing successive image portions of successive film layers and then repeating this process at multiple times while film 101 is developed.
For example, scanning a three-layer film 101 includes capturing reflected light 11 from a portion (e.g. portion 11a) of a first or xe2x80x9cfrontxe2x80x9d film layer 111 and capturing reflected light 12 from a corresponding portion (e.g. portion 12a) in a third or xe2x80x9cbackxe2x80x9d film layer 113. Such scanning also includes capturing a corresponding portion (e.g. portion 11a) in the second (i.e. xe2x80x9cmiddlexe2x80x9d or xe2x80x9cthroughxe2x80x9d) layer by scanning transmitted light 11 passing through film 101, and then subtracting scanned front layer and back layer values for corresponding front layer and back layer portion scans. A complete image dataset therefore includes front-layer, through-layer and back-layer image pixel data for each portion of the recorded image (in each film layer) at each scan-time during film development.
As shown in FIGS. 2A through 3, multiple scans at increasing times during film development (xe2x80x9ctimed-scansxe2x80x9d) are used to enable more accurate reproduction of a source image. Beginning with FIG. 2A, during picture taking with camera 210, a single image 201 recorded onto film 101 (FIG. 1) will typically include discernable picture elements such as highlights 211a, midtones 21b and shadows 211c. Turning to FIG. 2B, during development of film 101, an early scan 202a (e.g. one minute) will best reveal highlights 211a, while midtones 211b and shadows 211c will be under-developed. A laterscan (e.g. two minutes) will better reveal midtones 211b, while highlights 211a will become overdeveloped. Still later scans will better reveal shadows 211c at the expense of highlights 211a and midtones 211b. Thus, an image dataset comprising multiple timed-scans representing image elements (e.g. highlights, midtones and shadows covering a complete development time range is used to enable an accurate exposure of each element to be determined.
As shown in FIG. 3, a sufficient number of scans are taken such that the exposure of selected picture elements in each film layer can be deduced. Individual scans can further be combined for reducing memory utilization. (Scans 302 and 304, for example, can be combined to produce scan 308.)
FIGS. 3 through 5 illustrate examples of systems for combining image data contained in an image dataset (xe2x80x9cstitching systemsxe2x80x9d). A key goal of such systems is to form, from the timed-scans in an image dataset, a resultant image dataset wherein each image element is represented at the brightness or xe2x80x9cexposurexe2x80x9d at which it was recorded onto film. For example, in the system of FIG. 3 (hereinafter referred to as xe2x80x9csplice-stitchingxe2x80x9d) an approximation is made as to the best exposure for each group of picture elements utilized (i.e. in this case, highlights, midtones and shadows). In addition, the groups are aligned, cut and pasted to form a single image. Unfortunately, approximating exposure values (which might be between available timed-scan data), such that each element is properly exposed and a visually continuous single image is formed, has been found to be computationally expensive and to yield inconsistent results.
FIG. 4 illustrates an alternative stitching system, referred to as xe2x80x9cparametric-stitching,xe2x80x9d taught by a U.S. Patent Application entitled xe2x80x9cParametric Image Stitchingxe2x80x9d filed Feb. 22, 1999 and based upon Provisional Application No. 60/075,562, which application is assigned to the same assignee as that of the present invention. Captured pixels within an image dataset are preferably used as image elements for stitching purposes. As described, timed-scan pixel data is tagged with a capture-time (i.e. a relative and/or absolute indicator of the time of each scan during film developing). Further, for each pixel at each scan-time, regression parameters are calculated and. summed, the optimal density of each pixel position on the film is predicted, a gamma correction function is applied, and a brightness value for each pixel is determined. FIG. 4, for example, shows how different curves representing low, medium and high exposure timed-scans are preferably obtained for each pixel based upon a xe2x80x9cbest fitxe2x80x9d of received pixel data for each of the different timed-scans, and an optimum density curve is empirically derived, as shown by dotted line 404. The actual best xe2x80x9cbrightnessxe2x80x9d (or xe2x80x9cexposurexe2x80x9d) for a pixel is preferably determined based upon the intersection of the optimum density curve with the best fit curve corresponding.to the pixel type (i.e. whether it was a low, mid or high exposure pixel).
Among the advantages of parametric-stitching over splice-stitching is the replacement of xe2x80x9cpasting groups of picture elements togetherxe2x80x9d with a more reliable model-based implementation (i.e. where the proper exposure of a pixel can be resolved without reference to the image as a whole)
FIG. 5 illustrates a still further image stitching system as taught by U.S. patent application Ser. No. 09/196,208 filed Nov. 20, 1999 entitled xe2x80x9cLog-Time Processing and Stitching Systemxe2x80x9d and is taken from FIG. 5A of that application. As with parametric-stitching, captured pixels within an image dataset are preferably used as image elements for stitching purposes.
In one aspect of this system (hereinafter referred to as xe2x80x9clog-time stitchingxe2x80x9d), a regression analysis is performed that compares image data at various development times versus the natural log of time to obtain a best fit line of this data. The best-fit line is then used to determine a xe2x80x9cbxe2x80x9d value or xe2x80x9cfitting constantxe2x80x9d which preferably corresponds to the y-intercept of the best-fit line. It is discovered that this xe2x80x9cbxe2x80x9d value is substantially directly proportional to the log exposure of a corresponding pixel. As is taught in another aspect of log-time stitching, a xe2x80x9cbxe2x80x9d value can be calculated using the principles of matrix algebra according to the following equation-1                                                         [                                                                    N                                                                              ∑                                              ln                        ⁡                                                  (                          t                          )                                                                                                                                                                                ∑                                              ln                        ⁡                                                  (                          t                          )                                                                                                                                                                        ∑                                                  ln                          ⁡                                                      (                            t                            )                                                                                              2                                                                                  ]                        A                    ⁢                                    [                                                                    b                                                                                        m                                                              ]                        B                          =                              [                                                                                ∑                    S                                                                                                                    ∑                                          S                      ⁡                                              (                                                  ln                          ⁡                                                      (                            t                            )                                                                          )                                                                                                                  ]                    C                                    Equation  1            
wherein xe2x80x9cNxe2x80x9d represents a number of timed-scans made for a given film pixel (or given pixel of a single film layer in a multiple layer film), xe2x80x9ctxe2x80x9d represents each scan time relative to the start of film developing, xe2x80x9cSxe2x80x9d represents a detected signal value corresponding to the signal received from the sensor, xe2x80x9cmxe2x80x9d represents the slope of the best fit line, and xe2x80x9cb,xe2x80x9d which is referred to as the xe2x80x9cb value,xe2x80x9d is the y-intercept of the best fit line. Scan times can, as appropriate, be approximated as being equal for all pixels, such that matrix-A of equation-1 can be calculated once and then utilized for determining a xe2x80x9cbxe2x80x9d value for all pixels.
U.S. Pat. No. 5,519,510 and the Parametric Image Stitching application base upon U.S. Application Nos. 60/075,562 and U.S. application No. 09/196,208 are each expressly incorporated by reference herein.
While the above-described DFP capturing and stitching methods provide for more accurate image reproduction and/or processing of an image dataset, the resultant dataset has unfortunately been found to include a substantial amount of noise. More specifically, an undesirable signal distortion or xe2x80x9cnoisexe2x80x9d (e.g. which is not present in a source image) is found to be contained in the image data produced through DFP-based image capturing and stitching. Further, such noise is found to be visually apparent in a final image, appearing as blotches when the final image is viewed.
A conventional approach to reducing the noise content in image data is to filter the image data. However, such an approach is also found to unreliably distinguish between desirable image data and undesirable noise. Thus, despite positive effects of filtering in removing noise, filtering also unfortunately tends to remove source image information and can therefore ultimately impede the ability of the DFP system to accurately reproduce a source image.
Accordingly, there remains a need for a system capable of utilizing DFP captured image data to provide resultant image data having a reduced amount of noise without substantially reducing the amount of image information.
The present invention includes a system capable of utilizing DFP captured image data to provide resultant image data having a reduced amount of noise without substantially reducing the amount of image information. Broadly stated, a stitching and processing system according to the invention identifies a replaceable portion of an image in an image dataset having a relatively high noise content and then replaces such portion with a replacement portion from another image dataset having lower noise content. The at least two datasets are preferably taken from the same source image such that a resulting image is substantially visually unaltered by the replacement except for a reduction in noise content.
In a preferred embodiment, at least two sets of timed-scan image data are received from a DFP image capturing system. More preferably, two image datasets are received which have been captured from the same source image. The first image dataset is hereinafter referred to as a xe2x80x9ccolor-stitched imagexe2x80x9d and the second image dataset is hereinafter referred to as a xe2x80x9cluminance imagexe2x80x9d. Preferably, the color-stitched image corresponds to a conventionally taken number of timed-scans of a source image which timed-scans preferably range from when the source image is very under-developed to very over-developed. Also preferably, the luminance image corresponds to fewer timed-scans ranging from when source image from normally developed to very over-developed. Both received image datasets are preferably stitched, and more preferably, according to the log-time stitching system discussed in the above Background of the Invention. The stitched images are further preferably converted to a color space wherein image color data is separated from image non-color data, and more preferably, to L*a*b*, YCrCb, or other similar color space. Following stitching and conversion, the non-color data (e.g. the L* or luminance channel using an L*a*b* color space) of the luminance image is combined with the color data (e.g. the a* and b* or chrominance channels using L*a*b* space) of the color-stitched image.
Advantageously, a system according to the invention operates to provide a stitched image having substantially lower noise and otherwise not significantly affected source image reproduction characteristics as compared with existing systems.